Data in the First Mile

In many disadvantaged communities worldwide, local low-resource organizations strive to improve health, education, infrastructure, and economic opportunity. These organizations struggle with becoming data-driven, because their communities still live outside of the reach of modern data infrastructure, which is crucial for delivering effective modern services. In this paper, we summarize some of the human, institutional and technical challenges that hinder effective data management in “first mile” communities. These include the difficulty of deploying, cultivating and retaining expertise; oral traditions of knowledge acquisition and exchange; and mismatched incentives between top-down reporting requirements and local information needs. We propose a set of directions, drawing from projects that we have implemented. They include 1) separating the capture of data from its structuring, 2) applying intelligent automation to mitigate human, institutional and infrastructural constraints, and 3) deploying services in cloud infrastructure, opening up further opportunities for human and computational value addition. We illustrate these ideas in action with several projects, including Usher, a system for automatically improving data entry quality based on prior data, and Shreddr, a hosted paper form digitization service. We conclude by suggesting next steps for engaging in data management problems in the first mile.